Science with a punch

There are changes afoot in the way that US granting agencies, most notably the NIH, evaluate proposals. Change can be a good thing, and it is probably high time that the NIH reassessed how grants are evaluated. Frankly, I was becoming blasé about the entire granting process. It seemed that once I became inured to the low probability of funding, competition from the mega-labs, and the shift towards preferring science that can be called ‘medically relevant’, all of the fun had been taken out of writing grants and reading the subsequent scathing evaluations. Now, the evaluation of science will include a more precise and specific ranking of several individual aspects of a proposal, one of which is the (real or imagined) significance or impact of the proposed research. Ah . . . life is worth living again! How can one argue with the notion that significant science should be funded and insignificant science should not? Although explicit ranking of the impact of the proposed research seems to be a great idea, surely reviewers have been overtly or covertly considering significance when rating grants all along. So why has this criterion now come out of the closet and how will it be applied evenly to all grant applications? One major reason for the change in the method of evaluation is that a mechanism for provoking reviewers into giving out a wider range of rankings seems to be needed, since so many grants score high and so few can be funded. If the evaluation can be broken down into smaller quanta there is a chance that the overall scores may have greater variability. But how can we put a number on such an inherently unpredictable aspect of science as its future significance? Yes, some projects have obvious impact, but one of the attractions of this business is the fact that there are times when biology is determined to undermine your nice, tidy, and oh-so-logical model in a new, fascinating and wholly unpredicted way. Apparently routine questions are forever coming up with unexpected answers. Will the new system reduce our ability to check and recheck the truth of our assumptions? Moving to the personal level, how can I assess the likely impact of my own research? How will I know that my work will affect the concepts or methods that drive my field, and how can I be sure that I am working on an important problem? I increasingly suspect that the fact that I think my science is interesting and important is irrelevant to the misguided people out there who don’t share my views. Is there a generally acceptable way to define high-impact science?